Data-driven photographic style using local transfer

After taking pictures, photographers often seek to convey their unique moods by altering the style of their photographs, which can involve meticulous contrast management, lighting, dodging, and burning. In this sense, not only are advanced photographers concerned about their pictures’ styles; casual photographers who take pictures with cellphone cameras also process their pictures using built-in applications to adjust the image’s luminance, coloring, and details. In general, photographers who stylize pictures give them new, different visual appearances, while also preserving the original content. In this context, we investigate problems with novel image stylization, including reproducing the precise time-of-day where the lighting and atmosphere can make a landscape glow, and making a portrait style resemble that created by a renowned photographer. Given an already captured image, however, automatically achieving given styles is challenging. In fact, changing the appearance in a photograph to mimic another time-of-day requires the analysis and modeling of complex 3-D physical light interactions in the scene, while reproducing a portrait photographer’s unique style require computers to acquire artistic tastes and a glimpse of the artist’s creative process. In this dissertation, we sidestep these AI-complete problems to instead leverage the power of data. We exploit an image database consisting of time-lapse data describing variations in scene appearance 3 during the course of an entire day, and stylish portraits that are already deliberately processed by artists. To leverage these data, we present new algorithms that put input images in dense and local correspondence with examples. In our first method, we change the time-of-day with a single image as the input, which we put in correspondence with a reference time-lapse video. We then extract the local appearance transformations between different frames of the reference, and apply them to the input. In our second method, we transfer the style of a portrait onto a new input by way of local and multi-scale transformations. We demonstrate our methods on public datasets and a large set of photos downloaded from the Internet. We show that we can successfully handle lightings at different times of day and styles by a variety of different artists. Thesis Supervisor: William T. Freeman Title: Professor of Electrical Engineering and Computer Science Thesis Supervisor: Frédo Durand Title: Professor of Electrical Engineering and Computer Science Thesis Reader: Wojciech Matusik Title: Professor of Electrical Engineering and Computer Science Thesis Reader: Sylvain Paris Title: Doctor, Research Scientist (Adobe Inc.)